fix(runtime): strip read-time decorations before the route-level seed apply's closed parse - #16162
Conversation
… apply's closed parse `POST /packages/:id/publish-drafts` reads each just-published `seed` body back through `protocol.getMetaItem` and hands it to `SeedLoaderRequestSchema`. That read exits through `decorateMetadataItem`, which stamps `_diagnostics` on every body whose type has a registered schema, and `SeedSchema` is closed — so the door refused the document it had just served, on a 200, as `seedApplied.error`: zero rows loaded and the author told their seed body failed spec validation. The direction is settled by the contract, not by judgement. `METADATA_READ_DECORATIONS` declares `_diagnostics` a key the read path derives and attaches to the response, and its module names "any re-parse of a served document" as a consumer that must strip. `_packageId` is deliberately NOT a member and `SeedSchema` accepts it via `MetadataProtectionFields` — measured on the real producer, the served body carries both and the schema refuses exactly one. So the fix is the declared helper, not a widened schema and not the export path's blanket underscore strip, which would drop provenance this schema allowlists on purpose. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01D47qPfEWVPmhguWgBZCi5N
…engine double Both are mechanical regenerations the gates asked for by name, not hand edits: - `check:check-system-context-census` reported pure LINE ROT — the three `domains/packages.ts` anchors on `content/docs/permissions/system-context.mdx` each moved by exactly the +9 lines this branch's import block added. Repaired with the gate's own `--fix`; it now reports OK over 105 sites and 140 anchors. - `check:engine-double-contract` reported the new test file's engine double as RETAINED-but-unrecorded on all three scanned verbs. Regenerated with `--write` (736 rows, 3 added, 0 lost), so the pin protects the file. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01D47qPfEWVPmhguWgBZCi5N
📓 Docs Drift Check
What this run could not see
Coarse fallback — 24 page(s) merely mention a changed package (the pre-#9192 predicate, kept for the deliberately-wide backstop): |
github-actions
Bot
added
documentation
Contract review (clause ②) — PASS on content · NOT LANDABLE (dirty) — PR #16162 at head
|
…ed-apply-decorated-envelope
…n/main The merge brought main's re-anchoring of `system-context.mdx` and this branch's own +9-line import block in `domains/packages.ts`. `merge=os-regen` resolved the page with exit 0 and no markers while silently keeping one side, so the page had to be regenerated from the merged tree rather than text-merged. `pnpm gen:system-context-census` (= `check-system-context-census.mjs --fix`) rewrote 3 anchors, all `domains/packages.ts`, all by exactly +9: `:241`->`:250`, `:274`->`:283`, `:543`->`:552`. Delta is line-anchor numbers only: with every digit run normalized the page is byte-identical before and after, at the same 423 lines and the same 65 rows. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01D47qPfEWVPmhguWgBZCi5N
Merge + regen lap done — new head
|
| commit | what |
|---|---|
fb8e018042a |
Merge remote-tracking branch 'origin/main' — parents 73cdcb34a76 and d5d8d50db2b |
5f05afb3365 |
chore(gates): re-anchor the system-context census after merging origin/main — the regeneration, as its own commit |
Delta
git diff --name-only 73cdcb34a76..5f05afb3365 is 387 paths, 385 of which are simply what origin/main brought in. The two readings that carry information:
Merge commit fb8e018042a, combined diff (only paths whose result differs from both parents, i.e. the real resolutions):
packages/runtime/src/domains/packages.ts
scripts/engine-double-contract.pinned.json
Regen commit 5f05afb3365:
content/docs/permissions/system-context.mdx
Branch contribution on top of main, git diff --name-only d5d8d50db2b..5f05afb3365 — the same 5 paths you audited, still 5:
.changeset/seed-apply-read-back-decorations.md
content/docs/permissions/system-context.mdx
packages/runtime/src/domains/packages-seed-apply-read-decorations.test.ts
packages/runtime/src/domains/packages.ts
scripts/engine-double-contract.pinned.json
No reviewed source moved
For each non-generated path I diffed the branch's own contribution before the merge (1f2a02ba0d6..73cdcb34a76) against after (d5d8d50db2b..5f05afb3365), ignoring only hunk headers and blob-hash lines:
IDENTICAL (40 +/- lines) packages/runtime/src/domains/packages.ts
IDENTICAL (432 +/- lines) packages/runtime/src/domains/packages-seed-apply-read-decorations.test.ts
IDENTICAL (15 +/- lines) .changeset/seed-apply-read-back-decorations.md
IDENTICAL (17 +/- lines) scripts/engine-double-contract.pinned.json
The symmetric check on the two resolved paths says main's side survived too, so both intents stack and neither was dropped:
MAIN-SIDE IDENTICAL (18 +/- lines) packages/runtime/src/domains/packages.ts
MAIN-SIDE IDENTICAL (7 +/- lines) scripts/engine-double-contract.pinned.json
Your line-anchor warning reproduced exactly
It reproduced, and the mechanism is now visible in the merge commit itself: content/docs/permissions/system-context.mdx is absent from the merge commit's combined diff. The merge=os-regen driver resolved it byte-for-byte to the branch's side. git merge-tree predicted no conflict, the merge reported none, and nothing in git said a side had been dropped. Only the gate did.
Exit codes below were captured by redirecting first, never by reading $? after a pipe.
Before regeneration — node scripts/check-system-context-census.mjs, exit 1:
::error::[site-without-a-row] packages/runtime/src/domains/packages.ts:250 reads `ec.isSystem` and NO row on the page anchors it
::error::[site-without-a-row] packages/runtime/src/domains/packages.ts:283 reads `ec.isSystem` and NO row on the page anchors it
::error::[site-without-a-row] packages/runtime/src/domains/packages.ts:552 reads `ec.isSystem` and NO row on the page anchors it
::error::[anchor-is-not-a-read-site] the page anchors packages/runtime/src/domains/packages.ts:241, which the census does not call an elevation read
::error::[anchor-is-not-a-read-site] the page anchors packages/runtime/src/domains/packages.ts:274, which the census does not call an elevation read
::error::[anchor-is-not-a-read-site] the page anchors packages/runtime/src/domains/packages.ts:543, which the census does not call an elevation read
check-system-context-census: 6 problem(s) over 140 anchors and 105 census sites.
Exactly the +9 your reading of the PR body predicted, and it verifies: 250−241, 283−274, 552−543.
Regeneration — pnpm gen:system-context-census, exit 0. One correction to your instruction, in your favour: this is not a second command. gen:system-context-census is node scripts/check-system-context-census.mjs --fix, so the two spellings you named are the same thing and the --fix was unconditional rather than conditional on rot.
re-anchored content/docs/permissions/system-context.mdx:166 `domains/packages.ts:241` -> `domains/packages.ts:250`
re-anchored content/docs/permissions/system-context.mdx:166 `:274` -> `:283`
re-anchored content/docs/permissions/system-context.mdx:163 `domains/packages.ts:543` -> `domains/packages.ts:552`
check-system-context-census --fix: 3 anchor(s) rewritten
After regeneration — node scripts/check-system-context-census.mjs, exit 0:
check-system-context-census: OK — 105 elevation read sites in 19 packages across 44 files, all anchored; 140 anchors resolve, 27 declared non-read; 19 row reference(s) resolve to their keyed row, 2 declared unheld.
pnpm check:system-context-census (self-test leg plus the check), exit 0:
check-system-context-census --self-test: all cases passed
check-system-context-census: OK — 105 elevation read sites in 19 packages across 44 files, all anchored; 140 anchors resolve, 27 declared non-read; 19 row reference(s) resolve to their keyed row, 2 declared unheld.
The pre-commit hook agreed on the way in, clearing the deferral marker:
✓ content/docs/permissions/system-context.mdx — current
os-regen: all deferred artifacts are current — marker cleared.
And the regen delta is line numbers and nothing else: with every digit run normalized the page is byte-identical before and after, at the same 423 lines and the same 65 table rows.
One family did not measure the first time
The first os-regen-merge.sh run stopped at step 3 with the census gate at exit 3, PREREQUISITE NOT MET — the dependency typescript is not installed — a fresh worktree has no node_modules and this gate imports typescript. That is NOT MEASURED: not a pass, not a failure, and not folded into any green count. pnpm install made it measurable and every reading above is from after that.
Landing
Not flipped, not enqueued, no auto-merge armed — still a draft, still yours and the PM seat's step.
mergeable_state is now blocked, no longer dirty.
One thing to know before you act on it: origin/main moved to 4e090ecdeff (#16128) while this lap ran, and that commit also edits scripts/engine-double-contract.pinned.json, which this branch edits. The new head is still clean against it — git merge-tree --write-tree origin/main 5f05afb3365 exits 0 with no driver notice, and I checked the predicted merge rather than trusting the exit code: the branch adds 1 ledger entry, main adds 1, and the merged tree keeps both with zero lost on either side. I did not merge it in; the obstruction you named is cleared, and chasing main further is outside a merge-only lap.
Generated by Claude Code
Contract review (clause ②) — PASS extended to head
|
Landing — provenance ·
|
| claim | how it was checked here |
|---|---|
| the regen commit is regen-only | get_commit 5f05afb3365 → exactly one file, content/docs/permissions/system-context.mdx, +17/−17 |
| no reviewed source moved across the merge | the PR's net diff vs main is unchanged: 5 files, +497/−3 before the merge and after it, with commits 2 → 4 — a different measurement from the round's own byte-identity proof, and it agrees |
system-context.mdx is absent from the merge commit's combined diff" |
NOT verified here. GitHub returned only the first 30 files for fb8e018042a and appears to be diffing against parent 1, so that claim remains the round's, not this seat's |
⇒ The landing does not rest on that third line. It rests on Lint & Repo Gates being green at this head — the job carrying check-system-context-census, which is the gate that caught the silent drop in the first place and the only instrument that ever said anything was wrong.
⭐ What this PR's merge lap taught, recorded because the next one will hit it
The merge=os-regen driver resolved the routed page byte-for-byte to the branch's side while reporting success: git merge-tree --write-tree exit 0, git merge no conflict and no marker, and roughly thirty of main's re-anchorings dropped. GitHub said dirty about the same snapshot only because it runs a plain text merge with no custom driver — the two probes answer different questions, and the locally-correct-looking one was the silently wrong one. The census gate found it: 6 problem(s) over 140 anchors and 105 census sites. The regen delta is then provably numbers-only — with every digit run normalised the page is byte-identical before and after, same 423 lines, same 65 rows. Posted as a measurement on card #15871, whose open PR states the corollary in the other direction only. ⛔ No wording proposed there; that card is governed and belongs to someone else.
Two errors in this seat's own dispatch, corrected for the next one
- ⛔ I wrote "run
pnpm gen:system-context-census, thencheck-system-context-census.mjs --fixif the census line rotted." They are the same command —package.jsondefinesgen:system-context-censusasnode scripts/check-system-context-census.mjs --fix, so the--fixis unconditional and there is no second step. I inherited the shape from the director's comment and added a conditional of my own without checking it. - ⛔ I wrote that
git diff --name-only old..newshould show no reviewed source file. Unsatisfiable as worded: the delta is 387 paths and necessarily includespackages/runtime/src/domains/packages.ts, because main itself edited that file. The round substituted a stronger check and was right to.
Generated by Claude Code
3 participants
Fixes #15591
The package-publish door's route-level seed apply could not consume the shipping protocol's own read-back envelope.
POST /packages/:id/publish-draftsreads each just-publishedseedbody back throughprotocol.getMetaItem, unwraps it, and hands it toSeedLoaderRequestSchema.safeParse. That read exits throughdecorateMetadataItem, which stamps_diagnosticson every body whose metadata type has a registered schema —seedhas one — andSeedSchemahas been closed since protocol 17. So the door refused the document it had just served, minted the refusal as a 422 and delivered it on a 200 asseedApplied.error: zero rows loaded, and the author told their seed body failed spec validation when nothing about it was wrong.Which side was wrong — decided by measurement, not by judgement
The card left the direction open (strip at the consumer vs. stop decorating at the producer) and warned that the two underscore keys are not one population. Driven against the real producer — a
state:'draft'seed row promoted by the shippingpublishPackageDraftson a realObjectStackProtocolImplementation, read back through the same protocol over the same engine — they are not, andspecnames which is which:_diagnosticsis a member ofMETADATA_READ_DECORATIONS, whose module states the rule this door was missing: such a key "belongs to the response, never to the document — a served body is therefore NOT a valid input to the schema that produced it until these are removed", and it names "any re-parse of a served document" as a consumer class that must strip. ⇒ the producer is correct; the consumer was skipping a declared step._packageIdis "deliberately NOT" a member — ADR-0010 envelope state, which "the closed metadata schemas allowlist … precisely so a served document keeps its provenance on re-parse".SeedSchemaspreadsMetadataProtectionFieldson purpose, and the measurement above confirms it parses clean.So the repair is
stripReadDecorationsat the unwrap — the same helper, for the same reason, thatrest-server.tsalready calls before parsing a serveddataset("A SERVED document is not a valid input to the schema that produced it"), thatservice-automation's cold-boot flow bind calls, and thatsaveMetaItemcalls before its verbatim persist. This makes the seed apply the fourth consumer of one declared mechanism rather than a new accommodation.⛔ Not a widened schema.
SeedLoaderRequestSchemaandSeedSchemaare untouched; no.passthrough(), no alias, no??fallback. Prime Directive #12 pushes back on a tolerant consumer, and this is not one: it removes an annotation the READ path added, not a key an author wrote.⛔ Not the blanket
startsWith('_')stripassemblePackageManifestruns 300 lines up in the same file. Triage was right that the precedent sits nearby and right that it must be answered rather than copied: a portable manifest must shed provenance, a re-parse must keep it. Reusing that rule here would drop_packageId/_provenancethis schema allowlists on purpose — measured, in §3's second ablation below.Is the fallback still live? Yes, and the spec says so
The card's first open question was whether to repair the fallback or delete it as unreachable.
PublishPackageDraftsResponseSchema's own note answers it without a judgement call: the REST door "back-fillsseedAppliedfor custom protocols that do not self-apply", declared as part of the wire face. Measured here too — the shipping protocol'spublishPackageDraftsdoes answerseedApplied(§0 asserts it), which is exactly why this path is invisible in the shipping composition and why the harness has to withhold that field to reach it at all. Deleting a declared behaviour was not available; hardening it was.Every consumer of the read-back envelope
Enumerated rather than fixed one call site (
stripReadDecorationscall sites plus everySeedSchema/SeedLoaderRequestSchemaconsumer):rest-server.ts— serveddatasetbeforeDatasetSchema.parseservice-automation— cold-boot flow bindmetadata-protocol—saveMetaItem's verbatim persistmetadata-diagnostics— re-parse before computing a verdictruntime/domains/packages.ts— the route-level seed applyplugin-email— effective template readThe other
SeedLoaderRequestSchemaparses (app-plugin.ts×3,marketplace-install-local-plugin.ts,metadata-protocol's ownapplySeedBodies) all read authored config, a manifest already cleaned on export, or a raw storeddraft.body— none is a served document, so none is affected.Tests
packages/runtime/src/domains/packages-seed-apply-read-decorations.test.ts, 6 cases. The publish and the read-back are both real; onlypublishPackageDraftsis a facade, and only because the fallback runs exclusively for protocols that do not self-apply.SeedLoaderService.prototype.loadis spied through, not replaced, so §3 observes the exact body the shipping loader received.The whole
src/domains/packages*set runs green beside it — 11 files / 169 tests passed, which includes the two suites that own this call site from the other side (packages-seed-apply-disclosure.test.tsandpackages-seed-apply-org-scope.test.ts), andpnpm --filter @objectstack/runtime typecheckpasses both halves. That second half is not vacuous here:packages/runtime/tsconfig.jsonexcludes**/*.test.ts, so the reading rests on the siblingcheck:test-typecheck --project tsconfig.test.json, andtsc --listFileson that project confirms the new file is in the program.Reverse verification, direction predicted in writing before running, mutation confirmed on disk both ways (anchored text counts plus blob hash) and restore proven by observed state (
git diff HEADempty, blob hash back to HEAD's) — never by an exit code:expected '[invalid_metadata] the published seed…' to be undefined.expected { object: 'project', …(4) } to have property "_packageId". This is the run that makes §3 load-bearing; without it the file would be satisfied by a strip that silently deletes provenance.No rebuild was required for either ablation and that is a property of the resolution, not an omission: the subject (
packages/runtime/src/domains/packages.ts) is reached from the suite through a relative specifier inside its own package, and@objectstack/specis aliased to source inpackages/runtime/vitest.config.ts. The imports that do resolve throughdist(@objectstack/metadata-protocol,@objectstack/metadata-core) are not mutated by either ablation, and the dependency closure was built before any of these readings.Gates
Union derived with
node scripts/pm/dispatch-gates.mjs --repo objectstack-ai/objectstack --commandsagainst the delivered diff and asserted against its ownReconciliation — 91 famil(ies)line (91 commands emitted). The whole union was re-run after the final commit, at73cdcb34a76. Exit codes captured before any pipe, in every case by redirecting first (cmd > log 2>&1; EXIT=$?) rather than reading$?after a pipe. The Artifact rosters block (36 further families) was run separately, also at73cdcb34a76. Two gates asked for a mechanical regeneration and got one, in their own second commit:check:system-context-census --fix(pure line rot — the threedomains/packages.tsanchors each moved by the +9 lines this branch's import block added) andcheck:engine-double-contract --write(the new test file's engine double, 736 rows, 3 added, 0 lost).NOT MEASURED, by name, each with its own verdict line and none read as a pass:
check:dual-build-cjs-loads— exit 3,PREREQUISITE NOT MET — this gate reads built output, and some package has no dist/.check:type-check-debt— exit 3, same class; it says explicitly that it is "NOT evidence that a recorded number went up".check:published-readme-exports— exit 3, same class.check:react-declaration-parity— needs a browser manifest dump; refuses rather than compute a false green.check:single-claim-paths— exit 2,NOT WIRED(noPR_NUMBERoutside CI). Its pnpm spelling resolves to--self-testonly, so that green grades the checker's fixtures, not this diff (Two artifact-roster gates report a green that is not PR clearance — the pnpm script names resolve to --self-test only #16030).check:partof-closing-keywordwas in the same bucket and was then measured for real: re-run with this body inPR_BODY, it exits 0 — "this PR carries no Part-of/closing-keyword contradiction". Its pnpm spelling is--self-testonly and is not what that reading rests on.Everything else in both blocks passed.
check:skill-examplesinitially refused for the same prerequisite reason and was measured after building@objectstack/client-react's closure: 257 prose examples type-check across 3 surfaces.Changeset
patchon@objectstack/runtime. A bug fix on a published package: no API is added, removed or changed, no spec surface moves, and no stored shape migrates — so neitherminornor a**BREAKING**banner with an ADR-0087 disposition marker applies (andcheck-changeset-no-majorrefusesmajorin the launch window regardless).🤖 Generated with Claude Code
https://claude.ai/code/session_01D47qPfEWVPmhguWgBZCi5N
Generated by Claude Code